Exactly solvable models of adaptive networks 
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A satisfiability (SAT-UNSAT) transition takes place for many optimization problems when the 
number of constraints, graphically represented by links between variables nodes, is brought above 
some threshold. If the network of constraints is allowed to adapt by redistributing its links, the 
SAT-UNSAT transition may be delayed and preceded by an intermediate phase where the structure 
self-organizes to satisfy the constraints. We present an analytic approach, based on the recently 
introduced cavity method for large deviations, which exactly describes the two phase transitions 
delimiting this adaptive intermediate phase. We give explicit results for random bond models 
subject to the connectivity or rigidity percolation transitions, and compare them with numerical 
simulations. 
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Unraveling the principles responsible for the structure 
of observed technological, sociological or biological net- 
works is currently an intensively pursued challenge 
Phase transitions occurring when the structure evolves 
are of particular interest. The simplest example is con- 
nectivity percolation i which takes place when the num- 
ber of links increases [2j . Analogous phase transitions are 
found in constraint satisfaction problems (CSP's) defined 
on random graphs, such as the A"- SAT or coloring prob- 
lems ■ They are referred to as SAT-UNSAT transitions 
and are related to the algorithmic complexity of solving 
these hard combinatorial problems . 

While past studies of SAT-UNSAT transitions have 
confined to random distributions of constraints 0, 0, 
it has recently been suggested that an additional phase 
transition could arise if the network is allowed to respond 
to the addition of constraints by reorganizing itself ■ 
In this scenario, an adaptive intermediate phase (AIP) 
is predicted where the system avoids the SAT-UNSAT 
transition and adopts a structure distinct from random 
graphs. In this letter, we consider models whose config- 
uration space consists of an ensemble of graphs, with a 
CSP defined on each graph, and we investigate the pres- 
ence of an AIP by solving these models analytically with 
the cavity method for large deviations 0. We thus pro- 
vide one of the very few analytical results available for 
exponential random graphs models, the general class of 
networks to which our models belong (see e.g. 0). 

We apply our general approach to rigidity percolation 
on random bond models; this is a family of CSP's initially 
designed to model network glasses, the physical system 
for which the presence of an AIP was first suggested 
and experimentally investigated [lfll ]. In the absence of 
adaptation, these materials are expected to undergo a 
rigidity transition when the mean coordination of their 
atoms is varied by modifying their composition |llL Il2l| . 
As recalled below, this rigidity percolation can be viewed 
as a particular example of SAT-UNSAT transition (lflj . 



Random bond models include models with a continu- 
ous connectivity percolation transition or a discontinu- 
ous rigidity percolation transition, which allows us to 
illustrate the necessity of a discontinuous SAT-UNSAT 
transition for observing an AIP. For rigidity percolation, 
we analytically describe this AIP and the related phase 
transitions by deriving new and presumably exact for- 
mulae for large deviation functions. 

Random bond models - Random bond models [T3 | 
are constructed from N point-like particles with d de- 
grees of freedom each, by adding successively and at 
random bonds between them. Each bond carries only 
a bond-stretching constraint, so that the addition of a 
bond can have two effects: if the distance between its 
two end-points is already constrained, the new bond does 
not modify the total number of degrees of freedom and 
is considered as redundant; in the opposite case, it sup- 
presses the degree of freedom associated with the relative 
distance between the two end-points. Globally, the num- 
ber of independent internal degrees of freedom, or floppy 
modes, can be written 



•A/floppy = dN - M + E r — d(d + l)/2, 



(1) 



where dN represents the total number of degrees of free- 
dom, M the number of bonds, E r the number of redun- 
dant constraints, and where d{d + l)/2 accounts for the 
global degrees of freedom in d dimensions. 

In the thermodynamical limit where N — > oo and 
M — > oo with fixed ratio a = M/N, the structure is a 
random graph with Poisson degree distribution |l "J| , and 
the density of floppy modes, n/ = A/floppy/-^, satisfies 



< = d — 



(2) 



with e r = E r /N. Two different phase transitions may 
occur when a is increased: rigidity percolation, when the 
largest rigid subgraph, with no floppy mode, becomes 
extensive; unstressed to stressed transition, when the 
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largest stressed subgraph, with a non-zero density of re- 
dundant constraints, becomes extensive. With random 
bonds, the two transitions occur in fact simultaneously 
at a critical value a c [Hi ] . 

An elementary counting argument, due to 
Maxwell |l7j . approximatively locates a c by bal- 
ancing the total number of constraints M with the total 
number of degrees of freedom, dN, yielding aMxi = d. 
The underlying assumption is that no redundant bond 
appears (e r = 0) before the number of floppy modes 
vanishes (rif = 0), which is not the case with random 
networks where the density of floppy modes never 
reaches zero (therefore a c < cvmxi)- 

Constraint satisfaction — Random bond models can 
be recasted in the broader framework of CSP's as fol- 
lows [lflj . An instance of the problem is given by a graph 
G. An arrow is assigned to each link of G, which must 
point toward one of the two adjacent sites, and the set of 
different orientations defines the set of admissible solu- 
tions. Such a configuration a is labeled by associating to 
each oriented link i — > j a Boolean variable a^j 6 {0, 1} 
with <Xj— >j — 1 if the arrow is directed from i to j (and 
crj^i = 1 — o~i—,j). The cost function is 

C G [a] = I max (0, d - J2 a i^) ~ ( d ~ ")] ( 3 ) 
i jei 

where j G i indicates that j is connected to i. Physically, 
an arrow means that the bond suppresses one degree of 
freedom to the node to which it points. By minimizing 
the cost function, we get the number of redundant con- 
straints for the graph G [l3j : 

E r [G] = min Cq [o~] ■ (4) 

Note that Cg[c] > for any er, so that E r [G] > as 
it should. We will focus on two particular cases: d = 2 
which describes 2d rigidity percolation, and d = 1, which 
describes the usual connectivity percolation. 

Infinite temperature transition — The formulation in 
terms of CSP allows to resort to the cavity method 01 
to determine the phase diagram as a function of the den- 
sity of constraints a, when G is taken at random from 
the ensemble Qffi of graphs with N nodes and M = aN 
links [l3j- This method indeed applies to any CSP de- 
fined on such random graphs, and has notably lead to 
the derivation of the phase diagrams of the K-SAT and 
coloring problems 0, 0. In contrast with these algo- 
rithmically hard CSP's, the present model can be solved 
by the simplest, "replica symmetric" [l9j . formulation of 
the cavity method. The density of redundant constraints 
thus obtained is consistent with previous studies |2p| | and 
reads 

e r = ^2 K-zanik) ( - - dj , r) = ^Tr 2ari (k), (5) 

k=d+l ^ ' k=d 
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FIG. 1: Density of redundant constraints e r for d = 2 as cal- 
culated from Eqs. 0; the symbols are from numerical simu- 
lations confirming that the negative part corresponds to the 
number of floppy modes on the 3-core. Inset: e r for d = 1. 

where ng(k) = e 6 /k\. A non-zero solution, 77 > 0, 
exists only for a > a\f\ This threshold corresponds 
to the percolation of the (d + l)-core [jflj. that is, the 
emergence of a sub-graph containing an extensive number 
of nodes all having connectivity at least (d+1) JIBj (only 
the percolation of the 2-core occurs simultaneously with 
connectivity percolation ^J). As shown in Fig.^ f° r d > 
2, when this non-zero solution appears, the corresponding 
e r is negative, and becomes positive only at ac > ctjf* . 

Such a negative prediction for an intrinsically posi- 
tive quantity is often indicative of the inadequacy of the 
replica symmetric assumption 0, Q- For the present 
problem however, we verified that no replica symmetry 
breaking could occur and found instead a simple inter- 
pretation for the calculated e r < 0: Eqs. JSJ) correspond 
to Maxwell counting on the (d+ l)-core. While Maxwell 
argument is only approximate for the complete graph, we 
propose that it is exact on the (d+ l)-core, with the on- 
set of stressed nodes coinciding with the disappearance 
of floppy modes; in other words, e r gives the number 
of floppy modes on the (d + l)-core when negative, and 
the number of redundant constraints when positive. We 
verified numerically this interpretation using the Pebble 
Game algorithm |21j, see Fig.^ 

It follows that the rigidity percolation on the complete 
graph occurs not at the (d + l)-core percolation thresh- 
old a^P but at the point where e r becomes positive 
(for d = 1, the two thresholds coincide). The same con- 
clusion was reached in earlier studies [lil l2cj , based on 
comparison with numerical simulations. 

Finite temperature - To analyze the consequences 
of allowing the network of constraints to adapt, we now 
consider the system whose configuration space is the en- 
semble of graphs with N nodes and M — aN links. 
To each graph G £ , we associate as energy the num- 
ber E r [G] of redundant constraints, as given by Eq. Q). 
The partition function is thus Zn(tj) = X)r>cr<°> e~ yEr ^ 
where the inverse temperature y allows to sample differ- 
ent subsets of Qp? \ with the infinite temperature limit 
y = corresponding to selecting the graphs irrespectively 
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FIG. 2: Rate functions L ca _ v (e r ) with d — 2 and a = 1.75, 
1.85, 1.95 and 2.05 (from left to right at the bottom), as 
obtained from Eqs. ©. The symbols are results of Monte 
Carlo simulations at fixed y. For a < a$ xl ], the branches 
with e r < must be replaced by L(e r < 0) = oo. For a — 

(2) 

1.75 < a,c , a part of the curve with e r > must also be 
substituted by a Maxwell construction (dotted line). 

of their energy, as considered previously. -Zjv(y) can be 
evaluated by computing the microcanonical entropy den- 
sity s(e r ) which gives through cxp[Ns(e r )] the number 
of graphs in with density of redundant constraints 
e r . An equivalent information is contained in the rate 
function L{e r ), defined from the probability ¥pj(e r ) for a 
graph in to have density of constraints e r ; indeed, 
if Pjv(e r ) satisfies a large deviation principle, PAr(e r ) x 
exp[— NL(e r )] (ajv X fr/v means that lnajv/hi&jv —> 1 as 
N — > oo), and, if \Q£ | denotes the number of graphs in 
we have exp[^s(e r )] = \g { ° ] \exp[-NL(e r )]. The 
equilibrium properties of the system are thus captured 
by the potential 4>(y), with 

e -Wv) = \g^\-^Z N (y) x J de r e»b«-*>M\ (6) 

from which by Legendre transform we get 

e r = d<f>(y)/dy, L(e r ) = -ye r + <f>(y). (7) 

This potential can be computed by the large deviation 
cavity method [t| , an extension of the cavity method to 
atypical graphs. For random bond models, it yields: 

<t>{y) = - In Z + 2a [l - (1 - n)e v e~ z ] - az, 

d-l 

Z = e W + J- r K2caie~" (*) (V^^ - l) , 

1=0 (8) 
z = In [e y + (1 - e y ) V 2 ] , 

d-1 
k=0 

Examples of rate functions L cav (e r ) obtained from the 
non-trivial solution (77 > 0) of these equations are dis- 
played in Fig. |21 for d — 2 and four representative val- 
ues of a; they are perfectly consistent with numerical 



y 



1.7 1.75 1.8 1.85 1.9 1.95 2 2.05 2.1 

a 

FIG. 3: Phase diagram for d = 2. The two vertical dashed 
lines are for = ac (0) and a^xi = 2, and the full line 
is for ai 2 '(i/). At any fixed y > 0, an intermediate phase is 
present between a^\0) and ai 2 '(t/). The symbols are numer- 
ical estimates for the locations of the two phase transitions. 



results. For a < a^, = 2, L cav (e. r ) extends to e r < 0. 
As we checked with numerical simulations, these unex- 
pected negative values have the same interpretation as in 
the typical case (y = 0). This suggests that the correct 
rate function L(e r ) is obtained from L cav (e r ) by truncat- 
ing the negative branch: L(e r ) = L cav (e r ) if e r > 0, and 
L(e r ) = oo if e r < 0. For d = 1, where L cav (e r ) never 
shows a negative branch, our formulae coincide with rig- 
orous results from the mathematical literature \22\ (we 
are not aware of any previous result for d > 2). 

Intermediate phase - The rate function L(e r ) com- 
pletely specifies the equilibrium properties of the system 
for any y. Geometrically, the relation y = —dL(e r )/de r 
indicates that e r (y) is the intersection of L(e r ) with the 
supporting line having slope — y (the line below L(e r ) 
touching L(e r ) in a single point). For a < ai \ we 
thus get e r (y) = 0. For a > a.c , the rate function 
has a decreasing part whose slope, by convexity, is ex- 
tremal at the lower edge, e r = 0, with value denoted 
y [d) (a) = -dL/de r (e r = 0+). If y < y^ l \a), which is 
necessarily the case for d = 1 where y£ (a) = oo, the 
density of redundant constraints is e r (y) > and the 
system is in an UNSAT phase. For d > 2 however, since 
j/c d ' is finite when ac < a < a^ xl , there is an adaptive 
intermediate phase (AIP) between where yi d \a) = 
and ai d \y) where yi d \a) = y. In this phase, the system 
maintains itself at the edge of the SAT-UNSAT transi- 
tion, with e r {y) = 0. The resulting phase diagram is 
presented in Fig. [3] for d = 2. 

The atypical, self-organized, nature of the graphs in 
the AIP can be quantified with p = a 2 / (2a), the ratio of 
the variance a 2 of the degree distribution, over its mean 
2a. Indeed, typical random graphs have a Poisson de- 
gree distribution for which p = 1, and a value p ^ 1 
is therefore indicative of atypical graphs. We computed 
p with the large deviation cavity method, and found, as 
shown in Fig. 0] that it presents two slope discontinu- 
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FIG. 4: Order parameter p — a 2 /(2a) for d — 2 when varying 

(2) 

a at fixed y = 0.5. From the entry in the AIP at ct c = 1.79, 
the networks are atypical (non-Poisson) random graphs with 
p < 1. The symbols with error bars are from Monte Carlo 
simulations. Inset: analytical and numerical results for e r (y). 



ities, at the two boundaries of the AIP; the density of 
redundant constraints e r in contrast is strictly zero until 
a reaches (y) ■ This result is in very good agreement 
with our numerical simulations, and clearly demonstrates 
that the transition at CKc is of a different, topological, 
nature than the SAT-UNSAT transitions taking place 
here at ai d \y) > a?' = «c (y = 0). 

Zero temperature - It is worth mentioning that the 
extreme case y = oo, which has been the focus of several 
numerical studies 0, appears here as a very singular 
limit. In this case indeed, the distinction made previously 
between yc being infinite or not is irrelevant since y 
itself is infinite. An asymptotic analysis of our solution 
reveals that when y = oo an AIP is present between ai^ 
and c%li f° r an Y value of d, including d = 1 for which 
it disappears as soon as y < oo. This suggests that in 
finite dimensional models, where the rigidity transition 
is thought to be continuous, the AIP could be confined 
to y = oo. An AIP at finite y may be present only as 
cross-over, which could also account for the experimental 
observations in network glasses. 

Discussion - CSP's such as if -SAT or coloring are 
known for displaying a clustered intermediate phase 
(CIP), also known as hard-SAT phase, where the set of 
solutions breaks into an exponential number of discon- 
nected components 0,0- This CIP, which, in contrast 
to the AIP, takes place prior to the UNSAT-SAT transi- 
tion, is however of a radically different character: it refers 
to a transition in the space of solutions associated to sin- 
gle, typical, graphs, and not to a transition in a graph 
space. We analyzed here random bond models, which we 
argued do not involve replica symmetry breaking, and 
thus have no CIP. CSP's with a CIP could however be 
tackled along the same lines to study the interplay be- 
tween the two kinds of intermediate phases. 

In conclusion, we presented a general analytical ap- 
proach for solving the equilibrium thermodynamics of 
models of adaptive networks. This leads us to the ex- 



act description of a new kind of topological phase transi- 
tion, which takes place in a configuration space made 
of graphs. Our findings confirm that the presence of 
a discontinuous SAT-UNSAT transition in the typical 
problem is essential for observing an adaptive interme- 
diate phase (AIP) The principles behind the self- 
organization of graphs found in the AIP differ notably 
from most of the mechanisms proposed so far to explain 
the non-random structure of networks. Our model in- 
deed belongs to the family of exponential random graphs, 
a still poorly understood class of networks Q. Our re- 
sults show that the cavity method for large deviations is 
a powerful tool to study at least some of these models. 
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